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Executive Summary 



In November 2005, the National Assessment Governing Board created the Ad Hoc Committee 
on Planning for NAEP 12 th Grade Assessments in 2009. The Committee’s task was to help give 
focus and direction to the Board’s deliberations in three policy areas: 

• Conducting assessments at the state level in 12 th grade 

• Reporting on 12 th grade student preparedness for college-credit coursework, 
training for employment, and entrance into the military 

• Improving 12 th grade school and student participation 

A primary consideration guided the Committee’s deliberations: NAEP’s strengths and 
limitations. Thus, the recommendations indicate both steps that should be taken and steps that 
should not be taken. 

Of the three policy areas addressed, 12 th grade student preparedness presents the greatest 
challenges. Accordingly, the preponderance of recommendations is devoted to this policy area. 

With the aim of helping the Governing Board address these three policy areas successfully, the 
Committee has attempted to be as specific as possible in presenting its recommendations. 

The Ad Hoc Committee’s recommendations follow below. 



12 th Grade State NAEP 



Recommendation: The National Assessment Governing Board and the National Center for 
Education Statistics should proceed with planning for the implementation in 2009 of state-level 
assessments in reading and mathematics at grade 12 on at least a pilot basis in 10 states. 



12 th Grade Student Preparedness 

Recommendation 1. If found to be technically, operationally, and economically feasible, 
National Assessment of Educational Progress reports should include statements about 12 th grade 
student preparedness, beginning with the reading and mathematics assessments to be conducted 
in 2009. 

Recommendation 2. The Governing Board should develop and implement a plan for setting a 
final policy definition of “12 th grade student preparedness,” deciding on the statements about 
“preparedness” to include in NAEP reports, and conducting associated research and validity 
studies. 

Recommendation 3. The term “12 th grade student preparedness” should be limited to 
postsecondary education and postsecondary training for occupations (including occupations in 
the military); it should not include “entrance into the military” as recommended by the NAEP 
12 th Grade Commission. 
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Recommendation 4. The definition of 12 th grade student preparedness should focus on 
academic qualification without remediation for postsecondary education and postsecondary 
training for occupations; it should not include non-academic personal attributes. 

Recommendation 5. The reporting of “12 th grade student preparedness” should be done in 
conjunction with the Governing Board’s achievement levels — Basic, Proficient, and Advanced- 
rather than by setting separate “preparedness” performance standards. The degree of 
“preparedness” of students whose achievement is in the range below Basic should be fully 
reported as well. 

Recommendation 6. The reporting of 12 th grade student preparedness in NAEP should be kept 
as simple as possible to promote public understanding, consistent with available validity 
evidence. 

Recommendation 7. The Governing Board should decide on ancillary information relevant to 
12 th grade student preparedness to include in NAEP reports, such as information from NAEP 
background questionnaires and student transcripts collected through the NAEP High School 
Transcript Studies. 



12 th Grade School and Student Participation 

Recommendation 1. The National Center for Education Statistics (NCES) should continue 
implementing its plan for improving 12 th grade school and student participation. 

Recommendation 2. The NCES plan should be amended as follows: 

a. In addition to communicating with NAEP participants as described in the NCES plan, 
contact should be made with the district superintendent regarding the planned 
administration of NAEP. 

b. As a courtesy, a letter should be sent to the president of the district school board to 
communicate information about the plans to administer NAEP in the district. 

Recommendation 3. The Governing Board, through the Committee on Standards, Design and 
Methodology, should monitor and evaluate the implementation of the NCES plan and determine 
the impact on school and student participation. 




Introduction 



In November 2005, the National Assessment Governing Board created the Ad Hoc Committee on 
Planning for NAEP 12 th Grade Assessments in 2009. The Committee was asked to present 
recommendations at the August 2006 Governing Board meeting to give focus and direction to the 
Governing Board’s deliberations in three policy areas: 

• Conducting assessments at the state level in 1 2 th grade 

• Reporting on 12 th grade student preparedness for college-credit coursework, training 
for employment, and entrance into the military 

• Improving 12 th grade school and student participation 

The Board’s deliberations in these areas were prompted initially by the March 2004 report of the 
National Commission on NAEP 12 th Grade Assessment and Reporting (NAEP 12 th Grade 
Commission). Despite the progress made through assignments to Governing Board standing 
committees, there was a sense that the efforts lacked coherence. Consequently, the Ad Hoc 
Committee was commissioned. 

The Governing Board set the year 2009 as the target for implementation. This provided a concrete 
goal for planning and instilled a sense of immediacy to the work ahead. 

The Committee met eight times, either by conference call or face-to-face. Throughout the 
Committee’s deliberations, the guiding consideration was NAEP’s strengths and limitations. Thus, 
the recommendations address both steps the Board should take and steps not to take. 

Of the three policy areas under consideration, 12 th grade student preparedness presents the greatest 
challenges. The Committee spent considerable time addressing “preparedness” and, accordingly, 
the preponderance of recommendations is devoted to this area. 

It is with the hope of assisting the Governing Board successfully address the three policy areas of 
interest that this report is respectfully submitted by the Ad Hoc Committee on Planning for NAEP 
12 th Grade Assessments in 2009. 

The Committee’s recommendations are presented on the pages that follow. 
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1 2 th Grade State NAEP , . , 

Recommendation: The National Assessment Governing Board and the National Center lor 
Education Statistics should proceed with planning for the implementation in 2009 of state- 
level assessments in reading and mathematics at grade 12 on at least a pilot basis in 10 states. 



The Ad Hoc Committee believes that reporting state level results at grade 12 will provide important 
information to the public about student achievement, a view shared by a wide range of policy 
makers. 



For example the NAEP 12 th Grade Commission stated that its most important recommendation was 
for NAEP to provide 12 th grade results at the state level. President Bush asked for increases to the 
NAEP budget for FY 2006 and FY 2007 to conduct state level assessments at grade 12 in reading 
and mathematics. Senators Lamar Alexander and Edward M. Kennedy introduced the American 
History Act, authorizing pilot state assessments in up to ten states in U.S. history and civics at 
arades 8 and 1 2. Congress requested the Governing Board to prepare a report on the feasibility ot 
conducting state level assessments in U.S. history and civics in grades 8 and 12. 



A key question considered by the Ad Hoc Committee was— “Given agreement to go forward with 
planning for 12 th grade state NAEP in 2009, and assuming that sufficient funding is available, how 
should the Board proceed in seeking participants?” 



Drawing on the lessons learned from the Trial State Assessment and the Trial Urban District 
Assessment, the Committee concluded that the most practicable approach would be to start by 
identifying the states that want the information and are willing to step forward voluntarily. There 
were indications of enough potential state interest to begin with at least a small-scale pilot. 

Informal feedback from Chief State School Officers in response to the NAEP 1 2 th Grade 
Commission Report suggested that 6-12 states would be willing to participate. A formal survey of 
Chief State School Officers was conducted in June 2006 to supply information for the report to 

Congress on the feasibility ofstate assessments in U.S. history and civics. Thirteen of 26 

responding Chief State School Officers indicated potential interest in having their states participate 
at grade 12 and twenty at grade 8. 

While some state chiefs express support, others are opposed. A general concern is that 12 th graders 
will not take low-stakes NAEP seriously, the results will therefore be inaccurate, and that more 
problems will result from participating than educational benefit. Steps to take to improve 2 grade 
student effort are described in the third set of recommendations in this report. 



The Ad Hoc Committee recommends that the Governing Board proceed with planning for 
implementation ofstate assessments at grade 12 in 10 voluntarily participating states m 2009 The 
Committee also notes that on June 13, 2006, the House Appropriations Committee approved the 
President’s requested increase of $4 million in FY 2007 for 12 th grade state NAEP in reading and 
mathematics in 2009. Therefore, if Congress ultimately supports the President’s request, the 
planning should be based on 50 participating states rather than 10. 
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12 th Grade Student Preparedness 

Recommendation 1. If found to be technically, operationally, and economically feasible, ^ 
National Assessment of Educational Progress reports should include statements about 12 
grade student preparedness, beginning with the reading and mathematics assessments to be 

conducted in 2009. 



Background , ., ,. ,~th 

The Ad Hoc Committee believes that the Governing Board should proceed with reporting on 12^ 

trade student preparedness in connection with the 2009 assessments. For most students, the 12 
grade is the primary transition point to college, training for employment, and entrance into the 
military, the main paths to adult pursuits. Thus, information from NAEP about 12 grade student 
preparedness would be important in informing policies relevant to the economic well being and 

security of the nation. 

As the NAEP 12 th Grade Commission members observed, it is no longer true that a high school 
education is sufficient preparation for civic obligations and to qualify for a job that could sustain an 
individual or family comfortably. 

“The earnings gap between those with only a high school diploma and those with 
postsecondary education and training has widened substantially and the 
technological/scientific, legal, and moral complexity of today’s public policy issues require 
more to be an ‘informed citizen’ in a democracy.” 

Because it is the only continuing source of representative national and possibly state 12 grade 
student achievement data, it is appropriate that NAEP attempt to report on “preparedness. 

However, this significant elaboration of NAEP’s time-honored practice of reporting on what 
students know and can do” may be one of the most difficult tasks ever faced by NAEP. 

Predictive statements about student achievement have not been made in NAEP reports. Whether 
NAEP can validly support such statements is not certain, particularly because NAEP does not 
produce individual student scores. Without individual student scores, there is no direct way of 
relating performance on NAEP to other indicators of “preparedness” of individual students, such as 
performance on college entrance and placement tests and on assessments designed to help 
determine qualifications for training for occupations. 

Designing useful validity studies will require imagination, creativity, and the involvement of willing 
partners. Many technical challenges lie ahead. A systematic plan is needed for addressing those 
challenges. 
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Recommendation 2. The Governing Board should develop and implement a plan for setting a 
final policy definition of “12 th grade student preparedness,” deciding on the statements about 
“preparedness” to include in NAEP reports, and conducting associated research and validity 
studies. 

Background 

Ultimately, whether statements about “preparedness” can be included in NAEP reports is a validity 
issue. The definition of “preparedness,” the content of assessments (i.e., test frameworks, 
specifications, and items), the process for setting achievement levels, and the statements about 
preparedness intended to be made in NAEP reports must all be aligned and mutually supportive. 
Evidence must be gathered to demonstrate this alignment. In addition, the statements about 
preparedness to be included in NAEP reports should be corroborated by external evidence. 

Some preliminary work has already begun. The Governing Board has been overseeing revisions to 
the 12 th grade reading and mathematics frameworks and specifications for 2009, designed to support 
statements about “preparedness.” External panels have been convened to help define preparedness 
for college and for training for employment. Experts have prepared commissioned papers on 
validity issues. 

The Ad Hoc Committee believes that, building on this preliminary work, the Governing Board 
should now spell out the necessary steps — and a timeline for completing those steps — to report on 
12 th grade student preparedness in comiection with the 2009 assessment (see Appendix A for an 
outline of such a plan). The major outcomes of these steps would include: 

• Adoption of an overall policy definition for 12 th grade student preparedness 

• Decisions on the intended statements about 12 th grade student preparedness to be 
made in NAEP reports 

• Approval of revisions to the policy definitions for Basic, Proficient, and Advanced 
achievement levels to reflect “preparedness” 

• Approval of preliminary content definitions for preparedness in 12 th grade reading 
and in 12 th grade mathematics (to be used in achievement level setting) 

• Review of the 12 th grade reading and mathematics frameworks, specifications, and 
items for consistency with the policy definitions, statements about “preparedness” 
intended for NAEP reports, and content definitions for the respective subjects 

• Design and conduct of validity studies to provide evidence about the supportability 
of statements about “preparedness” 

The central and most fundamental major outcome is adoption of an overall policy definition for 12 th 
grade student preparedness. The Ad Hoc Committee developed Recommendations 3, 4, and 5 to 
assist the Governing Board in defining the term “12 th grade student preparedness” as it would be 
used in NAEP. 
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Recommendation 3. The term “12 th grade student preparedness” should be limited to 
postsecondary education and postsecondary training for occupations (including occupations 
in the military); it should not include “entrance into the military” as recommended by the 
NAEP 12 th Grade Commission. 

Background 

The Ad Hoc Committee believes that the categories under 12 th grade student preparedness should be 
changed from those recommended by the NAEP 12 th Grade Commission — “college-credit course 
work, training for employment, and entrance into the military.” Instead, the Ad Hoc Committee 
recommends two categories — “postsecondary education” and “postsecondary training for 
occupations” — and that “entrance into the military” be eliminated as a separate category. 

The intent is to simplify the categories. By changing the focus to preparedness for training for 
occupations, the military and civilian spheres are melded into a single category. Accordingly, the 
terms “postsecondary education” and “postsecondary training for occupations” should be defined — 
relative to the 12 th grade subject being assessed under NAEP — to refer, respectively, to: 

1 . Freshman level credit bearing higher education coursework leading to a degree, and 

2. Training for occupations — whether in the civilian or military spheres — that 

a. according to the Department of Labor are likely to offer sufficient compensation 
potential to support a family of four and provide for career advancement; and 

b. do not require a bachelor’s degree but do require training beyond high school. 

Occupations in the military are vast in range and similar in prerequisites and responsibilities to like 
civilian occupations. The military has conducted a great deal of validity research on the 
relationship between scores on its aptitude tests and preparedness for training for occupations in the 
military. This research can be drawn upon in developing and validating the statements about 
preparedness for training for occupations proposed for NAEP reports. 

There is debate about whether “preparedness for postsecondary education” and “preparedness for 
postsecondary training for occupations” are the same. ACT, Inc. has concluded that those entering 
college or workforce training programs after graduation “. . .need to be educated to a comparable 
level of readiness in reading and mathematics.” 1 Achieve, Inc. suggests there is a convergence of 
“. . .the English and mathematics that graduates must have mastered by the time they leave high 
school. . .to succeed in high-performance, high-growth jobs.” 2 On the other hand, Paul Barton 
provides evidence to argue against the proposition that “. . . [all] those not going to college need to 
be qualified to enter [traditional academic] college credit courses in order to enter the workforce.” 3 

Whether NAEP will be able to support statements about 12 th grade student preparedness for either 
postsecondary education or postsecondary training, for both, or for neither will need to be 
determined through research and validity studies as described in Recommendation 2, above. 



1 “Ready for College and Ready for Work: Same or Different?” ACT, Inc., Iowa City, IA; 2006. 

2 “Ready or Not: Creating a High School Diploma that Counts.” Achieve, Inc., Washington, D.C.; 2004 

3 “High School Reform and Work: Facing Labor Market Realities.” Paul Barton. Educational Testing Service, 
Princeton, New Jersey; 2006. 
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Recommendation 4. The definition of 12 th grade student preparedness should focus on 
academic qualification without remediation for postsecondary education and postsecondary 
training for occupations; it should not include non-academic personal attributes. 

Background 

Academic qualification: This recommendation emphasizes “qualification to enter” rather than 
“success in” or “completion of’ postsecondary education and training. This is because the 
measurement of achievement will occur at the end of high school — the “transition point” to adult 
pursuits — and the objective is to inform the public about the degree to which 12 th graders are ready 
academically for the next step, whether that next step is college or training for an occupation. 

NAEP is designed to measure student academic achievement and does this very efficiently. 
However, NAEP is not designed to measure non-academic characteristics often associated with 
success (whether in college or in the workplace) such as persistence, inter-personal communication, 
punctuality, and working in groups. Nor is NAEP designed to follow students longitudinally to 
determine whether postsecondary education or training was completed. Therefore, these factors are 
not included in the definition of 12 th grade student preparedness proposed for NAEP. 

Without remediation: The NAEP 12 th Grade Commission found that large percentages of college 
freshmen are being placed into non-credit remedial programs in reading, writing, or mathematics 
due to low scores on college entrance or placement exams (about 30 percent nationally and as high 
as 50 percent or more in some institutions). A recent research study by ACT, Inc. found that “only 
51 percent of. . .high school graduates [who took the ACT in 2005] are ready for college-level 
reading. . .” 4 

According to NCES, the college non-completion rate for students placed into any remedial program 
is very high. Of students who were 12 th graders in 1992, attended a public 2-year or 4-year college 
by 2000, and were placed into any remedial course, 58 percent did not complete a degree or receive 
a certificate. 5 This suggests that, although admitted to college, students requiring remediation in 
reading, writing, or mathematics are not sufficiently prepared academically to accomplish the tasks 
that will be required of them. 

Similarly, the National Association of Manufacturers, reporting on a 2001 survey of member 
companies on reasons they reject job applicants, found that 32 percent of respondents cited 
inadequate reading/writing skills and 21 percent cited inadequate math skills (N.B. 69 percent cited 
inadequate basic employability skills such as attendance and timeliness and 34 percent cited 
inadequate work experience, the first and second most prevalent reasons cited). 6 

Therefore, “without remediation” is a key factor in defining 12 th grade student preparedness. 



4 “Reading Between the Lines: What the ACT Reveals About College Readiness in Reading.” ACT, Inc.; 2005. 

3 “The Condition of Education 2004.” U.S. Department of Education, National Center for Education Statistics. 
Washington, DC. 

6 “The Skills Gap.” National Association of Manufacturers; Washington, D.C.; 2001. 
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Recommendation 5. The reporting of “12 th grade student preparedness” should be done in 
conjunction with the Governing Board’s achievement levels — Basic, Proficient, and 
Advanced — rather than by setting separate “preparedness” performance standards. The 
degree of “preparedness” of students whose achievement is in the range below Basic should be 
fully reported as well. 

Background 

The Ad Hoc Committee believes that “preparedness” should be reported in the context of 
achievement levels. This will conform more with reporting at grades 4 and 8, be less confusing to 
the public, and may provide a lower threshold for validity than if new, separate preparedness levels 
were set. Statements in NAEP reports about 12 th grade student preparedness would be used to help 
explain the meaning of the achievement level results. Accordingly, the policy definitions of 
“Basic,” “Proficient,” and “Advanced” at the 12 th grade would need to be revised to address 
“preparedness.” 

The revisions of the achievement level policy definitions should be informed by research. For 
example, research might be conducted to examine the relationship between scores on 12 th grade 
NAEP in reading and mathematics and cut scores on widely used tests for college entrance and 
placement. Similar research might be conducted using tests that assess qualifications for training 
for various occupations, such as the aptitude tests used by the military. Likewise, research could be 
conducted comparing the test content of NAEP and the other tests. The results would be used to 
help define the Basic, Proficient, and Advanced achievement levels at the 12 th grade. 

In addition, the results might be used to set “benchmark” scores on the NAEP scales for reading and 
mathematics. The benchmarks would represent preparedness for postsecondary education and 
postsecondary training for occupations. If indicated as necessary by the research, multiple 
benchmarks would be used to represent, for example, “preparedness” at colleges of varying 
selectivity and for training for occupations with varying academic prerequisites. 

The benchmarks could be displayed in association with item results, as in the current NAEP item 
maps. Showing the types of tasks that students at or near the benchmarks were likely to do 
successfully would add further meaning to the NAEP achievement level results. Figure 1 illustrates 
this approach, with the familiar item map to the right of the vertical NAEP scale and preparedness 
benchmarks to the left. 

The Ad Hoc Committee believes that it is essential to provide information about performance 
in the range below Basic. Of 12 th graders, approximately 26 percent are below Basic in reading 
and about 35 percent in mathematics. These students are likely to graduate from high school, yet 
their low levels of achievement suggest that their prospects immediately upon graduation will be 
limited in comparison to those of higher achieving students. 

The number of students achieving below the Basic level is significant. It is important to report as 
fully as possible on the preparedness of these students for postsecondary endeavors, as well as those 
performing in the Basic, Proficient, and Advanced achievement levels ranges. Because the 12 th 
grade assessment includes only youth in school, but not dropouts or other out of school youth, it 
also will be important to report clearly on which students are included in the sample and which 
students are not. 
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Figure 1. 



12 th Grade NAEP Scale and Achievement Levels 

(Example— Not NAEP Data) 



Preparedness Benchmarks Item Map (Mathematics) 



Preparedness benchmarks would be 
placed on this side of the NAEP scale. 

The specific benchmarks and their 
location along the scale would be 
determined by research. They would 
indicate the “qualifying” level, without 
remediation, for various postsecondary 
pursuits, which might include: 

1 . colleges of varying selectivity 
(e.g., community college, 4-year open 
admissions university, selective univer- 
sity, highly selective university, etc.) 
and 

2. training for occupations with 
varying academic prerequisites 
(e.g., laboratory technician, paralegal 
assistant, cosmetician, computer 
technician, electrician, air traffic 
controller, etc.). 




0 
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Recommendation 6. The reporting of 12 th grade student preparedness in NAEP should be 
kept as simple as possible to promote public understanding, consistent with available validity 
evidence. 

Background 

The objective of reporting on 12 th grade student preparedness is to enhance public understanding 
about the meaning of student achievement. The concept of 12 th grade student preparedness seems 
simple to understand and likely to add value to NAEP reporting, answering the question — Are 12 th 
graders ready to enter college or training for occupations without the need for remediation? 
However, this apparently simple idea may turn out to be overly complex in its implementation. 

There is a debate about whether preparedness for postsecondary education requires the same 
proficiencies in reading and mathematics as preparedness for postsecondary training for 
occupations. Some argue that the proficiencies are the same. However, others argue that colleges 
have varying standards for placement into credit-bearing coursework and that occupations for which 
training is required have differing prerequisite academic demands. 

If research supports the contention that the academic proficiencies needed to be prepared are the 
same for postsecondary education and for postsecondary training for occupations, reporting should 
be relatively straightfoiward and simple. However, the research might indicate that there are 
multiple “preparedness” levels, or that subtle caveats are required. The question then would be 
whether NAEP can reflect this variability faithfully and in a manner understandable to the public. 
Figure 1 above suggests a possible solution, but the answer lies in the research to be conducted. 

In the end, reporting on 12 th grade student preparedness should be done in a manner that will 
promote understanding on the part of the public. Whether to report and what to report will have to 
be determined by the Governing Board in light of the validity evidence and a judgment about what 
will be meaningful and comprehensible to NAEP’s audience. 



Recommendation 7. The Governing Board should decide on ancillary information relevant to 
12 th grade student preparedness to include in NAEP reports, such as information from NAEP 
background questionnaires and student transcripts collected through the NAEP High School 
Transcript Studies. 

Background 

In addition to achievement on NAEP, other information related to academics should be included in 
NAEP reports relevant to 12 th grade student preparedness. For example, data could be collected to 
address questions such as 

• What percentage of 12 th grade students have completed a “college prep” program of 
study? 

• What percentage of 12 th grade students pursued a vocational program and what types 
of vocational programs were offered? 

• What percentage of students participated in joint enrollment programs with local 
colleges? 

• Is there a correlation between attendance and achievement? 
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• What is the degree of access to Advanced Placement courses and International 
Baccalaureate programs? 

• Are there differences in course taking patterns by gender and by race/ethnicity? 

• Do courses with similar titles have similar curricular content or are differences found 
by some variable, such as region, type of school, or demography? 

• What percentage of students has taken college entrance exams? 

This information could be collected through the NAEP High School Transcript Studies and through 
NAEP background questionnaires. The NAEP High School Transcript Studies are currently 
scheduled once every four years, in connection with NAEP 12 th grade mathematics and science 
assessments. The next transcript study is scheduled for 2009, the target year for reporting 12 th grade 
student preparedness. NAEP background questionnaires are a component of each assessment. The 
Governing Board should determine the ancillary information to be reported in connection 
with the 2009 12 th grade assessments, specifying the information to be collected through 
NAEP background questionnaires and through the NAEP 2009 High School Transcript 
Studies. 



12 th Grade School and Student Participation 

Recommendation 1. The National Center for Education Statistics (NCES) should continue 
implementing its plan for improving 12 th grade school and student participation. 

Recommendation 2. The NCES plan should be amended as follows: 

a. In addition to communicating with NAEP participants as described in the NCES plan, 
contact should be made with the district superintendent regarding the planned 
administration of NAEP. 

b. As a courtesy, a letter should be sent to the president of the district school board to 
communicate information about the plans to administer NAEP in the district. 

Recommendation 3. The Governing Board, through the Committee on Standards, Design and 
Methodology, should monitor and evaluate the implementation of the NCES plan and 
determine the impact on school and student participation. 

Background 

The National Center for Education Statistics has proposed— and begun implementing— initiatives to 
improve school and student participation. These initiatives include: 

• Early notice to schools of their selection for the sample (spring of the school year 
prior to the assessment rather than fall of the school year in which the assessment is 
to be conducted) 

• Communication by NAEP state coordinators with high school principals about the 
reasons to participate 

• Placement of the assessment date on the school calendar for the next school year 

• Greater flexibility to the schools in scheduling the assessment 

• More persuasive materials about NAEP for principals, teachers, students and parents 
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These initiatives are described in more detail in Appendix B. NCES should continue to implement 
these initiatives. In addition, NCES should report periodically to the Board on the implementation 
of these initiatives and their impact on improving participation at the 12 th grade. The Committee on 
Standards, Design and Methodology (COSDAM) has offered to monitor implementation of the 
NCES plan on the Board’s behalf. The Ad Hoc Committee commends COSDAM for making this 
offer and recommends its acceptance formally by the Governing Board. 



Conclusion 

The future of 12 th grade NAEP is at a crossroad — it may flourish as a source of important 
information for the public or slowly wither into oblivion. The NAEP 12 th Grade Commission was 
created because low participation rates and concerns about student motivation raised questions 
about the very viability of 12 th grade NAEP. The Commission members concluded that 12 th grade 
NAEP had valuable, unrealized potential and that tapping that potential would invigorate NAEP at 
the high school level. 

The Ad Hoc Committee agrees. Reporting on 12 grade student preparedness will increase the 
relevance and usefulness of NAEP. Supplying state level results at the 12 th grade will provide 
information about student achievement that is otherwise unavailable. Improving 12 th grade school 
and student participation is essential if NAEP 12 th grade results are to be viewed with credibility. 
The Ad Hoc Committee hopes that the recommendations in this report are helpful toward these ends 
and that the Governing Board will act on them with due deliberation. 
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Appendix A 



2006 



2007 



Reporting 12 th Grade Student Preparedness in 2009: 
Example of Timetable of Key Events 

Receive Ad Hoc Committee Report 

Review of draft “preparedness” policy definition with 
stakeholders 

Review of 12 th grade reading and mathematics 
frameworks, specifications, and items in light of draft 
policy definition for preparedness 

Design of research and validity studies 

Develop draft statements about 12 th grade student 
preparedness to be considered for NAEP reports 

Develop draft policy definitions for 12 th grade achievement 
Levels (Basic, Proficient, and Advanced) 

Board standing committees consider technical and policy 
considerations 

Board makes decision about process for external comments 
on draft policy definition for 12 th grade student preparedness 

Public hearings, forums, Federal register notice, etc. to 
obtain comments about 12 th grade student preparedness: draft 
definition, policy and technical issues, etc. 

Revised policy definitions of “preparedness” and of Basic, 
Proficient and Advanced achievement levels at 12 th grade 
presented to Board for discussion 

Status report on reviews, development of research and 
validity studies, draft statements for reports, etc 

Begin work on preliminary content definitions for 
“preparedness” in 12 th reading and mathematics 

Board action on policy definition of “12 th grade 
preparedness” and 12 th grade achievement levels 
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Appendix A 



2007 Cont’d Approve preliminary content definitions for “preparedness” 

in 1 2 th grade reading and mathematics 

Approve proposed statements about preparedness for NAEP 
reports 

Conduct construct validity studies in 12 grade reading and 
mathematics 

Present results of construct validity studies; revise 
preliminary content definitions and statements for reporting 
accordingly 

2008 Gather validity evidence 

Award achievement level setting contracts 

2009 Administer reading and mathematics assessments 

Conduct achievement level setting activities 

Board action on 12 th grade student preparedness reporting 

2010 Report on 12 th grade student preparedness 
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materials Maintain operations, Created a brochure about how to find Will share the Private Schools report 

Provide information, private school results on the NAEP with the schools selected for 

Gain cooperation, website participation (2007) 

Create buy-in 
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Appendix C 



August 2002 
November 2002 

March 2003 
March 2004 
March 2004 

March 2005 

November 2005 
January 2006 

February 2006 
August 2006 



NAEP 12™ Grade Timeline 



Roy Truby’s final report to the Board — raises the problem of 12 th 
grade participation and motivation 

Governing Board approves creation of the National Commission 
on NAEP 12 th Grade Assessment and Reporting 

Co-Chairs: Mark Musick and Michael Nettles 



First Commission meeting 

NAEP 12 th Grade Commission presents report to NAGB 

NAGB makes assignments to Board Committees and creates the 
Ad Hoc Committee on NAEP 12 th Grade Participation and 
Motivation 

Chair: David Gordon 



Ad Hoc Committee on Participation and Motivation presents initial 
recommendations to NAGB 

NAGB embraces the recommendations and begins to address them 
through assignments to Board committees 

Board acts affirmatively on some recommendations 

Sense that there is a lack of coherence — approval for a committee 
to lend focus and direction to 12 th grade state NAEP 

Members appointed to Ad Hoc Committee on Planning for NAEP 
12 th Grade Assessments in 2009 

Chair: Sheila Ford 



Ad Hoc Committee on Planning: First meeting — organizational — 
by conference call; seven additional meetings 

Ad Hoc Committee presents recommendations to NAGB 

Work to be continued. . . 
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